Image processing device, image processing method, and computer-readable recording medium

ABSTRACT

An image processing device includes a processor including hardware. The processor is configured to: identify an image selected according to a user input operation received by an input device, from among a plurality of first images acquired by a capsule endoscope, as an identified image; determine whether or not the identified image is one of a plurality of second images acquired during a reciprocating movement of the capsule endoscope; and in response to the identified image being determined to be one of the plurality of second images, extract an image acquired in a first forward path or a last forward path of the reciprocating movement from among the plurality of second images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT international application Ser.No. PCT/JP2015/062741 filed on Apr. 27, 2015 which designates the UnitedStates, incorporated herein by reference, and which claims the benefitof priority from Japanese Patent Application No. 2014-193104, filed onSep. 22, 2014, incorporated herein by reference.

BACKGROUND

1. Technical Field

The disclosure relates to an image processing device, an imageprocessing method, and a computer-readable recording medium, whichdisplay an image acquired by a capsule endoscope that is introduced intoa subject and captures an image inside the subject.

2. Related Art

In recent years, in the field of endoscope, an examination using acapsule endoscope that is introduced into a subject such as a patientand captures an image is known. The capsule endoscope is a device wherean imaging function, a wireless communication function, and the like areincluded in a capsule-shaped casing formed into a size that can beintroduced into a digestive tract of the subject. The capsule endoscopesequentially and wirelessly transmits image data generated by capturingan image inside the subject to the outside of the subject. The imagedata wirelessly transmitted from the capsule endoscope is onceaccumulated in a receiving device provided outside the subject, istransferred (downloaded) from the receiving device to an image displaydevice such as a workstation, and is variously image-processed in theimage display device.

When observing a series of images acquired in this way, normally, a usersuch as a doctor sets an image including a feature portion such as anorgan boundary and a surgical scar as a feature point while generallybrowsing the images by reproducing the images as a moving image orcontinuously reproducing still images. Thereafter, the user performsdiagnosis such as extracting a desired image based on the set featurepoint and observing the image in detail or determining a motor functionof an organ based on a difference between imaging times corresponding toa moving time of the capsule endoscope from one feature point to anotherfeature point among a plurality of set feature points. Therefore, toperform a correct diagnosis, it is important to set feature points to anappropriate image.

As a technique related to extracting an image from a series of imagesacquired by a capsule endoscope, Japanese Laid-open Patent PublicationNo. 2006-280792 A discloses a technique to calculate a correlation valueof a plurality of pixel areas that are set on each image of a series ofimages, to calculate a motion vector of the plurality of pixel areas, todetect a continuous image group in which images having a correlationvalue of a plurality of pixel areas between adjacent images is greaterthan or equal to a predetermined value, continuously appear, to identifyone or more representative images from among the continuous image group,and to display the one or more representative images at a display framerate different from that of images other than the one or morerepresentative images.

SUMMARY

In some embodiments, an image processing device includes a processorincluding hardware. The processor is configured to: identify an imageselected according to a user input operation received by an inputdevice, from among a plurality of first images acquired by a capsuleendoscope, as an identified image; determine whether or not theidentified image is one of a plurality of second images acquired duringa reciprocating movement of the capsule endoscope; and in response tothe identified image being determined to be one of the plurality ofsecond images, extract an image acquired in a first forward path or alast forward path of the reciprocating movement from among the pluralityof second images.

In some embodiments, an image processing method includes: identifying animage selected according to a user input operation, from among aplurality of first images acquired by a capsule endoscope, as anidentified image; determining whether or not the identified image is oneof a plurality of second images acquired during a reciprocating movementof the capsule endoscope; and in response to the identified image beingdetermined to be one of the plurality of second images, extracting animage acquired in a first forward path or a last forward path of thereciprocating movement from among the plurality of second images.

In some embodiments, a non-transitory computer-readable recording mediumrecording an image processing program is provided. The program causes animage processing device to execute: identifying an image selectedaccording to a user input operation, from among a plurality of firstimages acquired by a capsule endoscope, as an identified image;determining whether or not the identified image is one of a plurality ofsecond images acquired during a reciprocating movement of the capsuleendoscope; and in response to the identified image being determined tobe one of the plurality of second images, extracting an image acquiredin a first forward path or a last forward path of the reciprocatingmovement from among the plurality of second images.

The above and other features, advantages and technical and industrialsignificance of this invention will be better understood by reading thefollowing detailed description of presently preferred embodiments of theinvention, when considered in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a schematic configuration ofa capsule endoscope system including an image display device accordingto a first embodiment of the present invention;

FIG. 2 is a diagram illustrating a schematic configuration of a capsuleendoscope and a receiving device illustrated in FIG. 1;

FIG. 3 is a block diagram illustrating a schematic configuration of theimage display device illustrated in FIG. 1;

FIG. 4 is a schematic diagram illustrating an example of a positionseries index created by an index creation unit illustrated in FIG. 1;

FIG. 5 is a flowchart illustrating processing that transfers image datafrom the receiving device illustrated in FIG. 1 to the image displaydevice;

FIG. 6 is a flowchart illustrating feature point setting processingperformed by the image display device illustrated in FIG. 1;

FIG. 7 is a schematic diagram illustrating a display example of a seriesof images acquired by the capsule endoscope illustrated in FIG. 1;

FIG. 8 is a schematic diagram illustrating a display example of acandidate image;

FIG. 9 is a schematic diagram illustrating an example of an image numberindex;

FIG. 10 is a schematic diagram illustrating an example of a time seriesindex;

FIG. 11 is a flowchart illustrating feature point setting processingperformed by an image display device according to a second embodiment ofthe present invention;

FIG. 12 is a block diagram illustrating a configuration of an imagedisplay device according to a third embodiment of the present invention;

FIG. 13 is a flowchart illustrating feature point setting processingperformed by an image display device according to the third embodimentof the present invention.

FIG. 14 is a block diagram illustrating a configuration of an imagedisplay device according to a fourth embodiment of the presentinvention; and

FIG. 15 is a flowchart illustrating organ function determinationprocessing performed by the image display device illustrated in FIG. 14.

DETAILED DESCRIPTION

Hereinafter, an image display device, an image display method, and animage display program according to embodiments of the present inventionwill be described with reference to the drawings. The present inventionis not limited by the embodiments. In the drawings, the same componentsare denoted by the same reference numerals.

First Embodiment

FIG. 1 is a schematic diagram illustrating a schematic configuration ofa capsule endoscope system including an image display device accordingto a first embodiment of the present invention. A capsule endoscopesystem 1 illustrated in FIG. 1 includes a capsule endoscope 2 whichgenerates image data by being introduced into a subject 10 and capturingan image in the subject 10 and transmits the image data by superimposingthe image data on a wireless signal, a receiving device 3 that receivesthe wireless signal transmitted from the capsule endoscope 2 through areceiving antenna unit 4 attached to the subject 10, and an imagedisplay device 5 that receives the image data from the receiving device3, performs predetermined image processing on the image data, anddisplays an image.

FIG. 2 is a block diagram illustrating a schematic configuration of thecapsule endoscope 2 and the receiving device 3.

The capsule endoscope 2 is a device that includes various componentssuch as an imaging element in a casing having a capsule shape with asize that can be swallowed by the subject 10. The capsule endoscope 2includes an imaging unit 21 that captures an image inside the subject10, an illumination unit 22 that illuminates inside the subject 10, asignal processing unit 23, a memory 24, a transmitting unit 25, anantenna 26, and a battery 27.

The imaging unit 21 includes, for example, an imaging element such asCCD or CMOS that generates and outputs an imaging signal representingthe inside of the subject 10 from an optical image formed on a lightreceiving surface and an optical system such as an objective lensarranged on a light receiving surface side of the imaging element. Theillumination unit 22 is realized by a semiconductor light emittingelement such as LED (Light Emitting Diode) or LD (Laser Diode) lightsource that emits light toward the subject 10 during imaging. Thecapsule endoscope 2 includes a circuit substrate where a drive circuitor the like that drives each of the imaging unit 21 and the illuminationunit 22 is formed. The imaging unit 21 and the illumination unit 22 arefixed to the circuit substrate in a state in which the imaging unit 21and the illumination unit 22 face toward the outside from one endportion of the capsule endoscope 2.

In FIG. 2, the imaging unit 21 and the illumination unit 22 are providedin one end portion of the capsule endoscope 2, so that an image on oneside as seen from the capsule endoscope 2 can be captured. However, aplurality of imaging units 21 and illumination units 22 may be providedand a plurality of directions, for example, forward and backwarddirections or direct-viewing and side-viewing directions, may be definedas directions in which imaging can be performed.

The signal processing unit 23 controls each unit in the capsuleendoscope 2, performs A/D conversion of the imaging signal outputtedfrom the imaging unit 21 to generate digital image data, and furtherperforms predetermined signal processing on the image data.

The memory 24 temporarily stores various processing programs, modules,and routines, which are executed by the signal processing unit 23, andthe image data signal-processed by the signal processing unit 23.

The transmitting unit 25 and the antenna 26 superimpose the image datastored in the memory 24 on a wireless signal along with relatedinformation and transmit the wireless signal to the outside.

The battery 27 supplies power to each unit in the capsule endoscope 2.It is assumed that the battery 27 includes a power supply circuit thatboosts power supplied from a primary battery or a secondary battery suchas a button battery.

The capsule endoscope 2 is swallowed by the subject 10 and then thecapsule endoscope 2 sequentially captures images of biological regionssuch as esophagus, stomach, small intestine, and large intestine atpredetermined time intervals, for example, at time intervals of 0.5 secwhile being moved in the digestive tract of the subject 10 byperistaltic movement and the like of organs. The capsule endoscope 2sequentially and wirelessly transmits image data and related informationgenerated by the image capturing operation to the receiving device 3.The related information includes identification information such as, aserial number assigned to identify individual of the capsule endoscope2.

The receiving device 3 receives the image data and the relatedinformation wirelessly transmitted from the capsule endoscope 2 throughthe receiving antenna unit 4 including a plurality of (in FIG. 1, eight)receiving antennas 4 a to 4 h. Each of the receiving antennas 4 a to 4 his realized by using, for example, a loop antenna and is arranged to apredetermined position on an outside surface of the subject 10. As anexample, each of the receiving antennas 4 a to 4 h is arranged to aposition corresponding to an organ in the subject 10, which is a passageroute of the capsule endoscope 2.

As illustrated in FIG. 2, the receiving device 3 includes a receivingunit 31, a signal processing unit 32, a memory 33, a data transmittingunit 34, an operating unit 35, a display unit 36, a control unit 37, anda battery 38. The receiving unit 31 receives the image data wirelesslytransmitted from the capsule endoscope 2 through the receiving antennas4 a to 4 h. The signal processing unit 32 performs predetermined signalprocessing on the image data received by the receiving unit 31. Thememory 33 stores the image data on which signal processing is performedby the signal processing unit 32 and information related to the imagedata.

The data transmitting unit 34 is an interface that can be connected to acommunication line such as USB, wired LAN, or wireless LAN. The datatransmitting unit 34 transmits the image data and the relatedinformation stored in the memory 33 to the image display device 5, undercontrol of the control unit 37. The operating unit 35 is used when auser inputs various setting information and the like of the receivingdevice 3.

The display unit 36 displays registration information (examinationinformation, patient information, and the like) related to examinationand various setting information inputted by a user. The control unit 37controls operation of each unit in the receiving device 3. The battery38 supplies power to each unit in the receiving device 3.

While the capsule endoscope 2 performs imaging, in other words, during aperiod from when the capsule endoscope 2 is swallowed by the subject 10to when the capsule endoscope 2 is discharged, the receiving device 3 isattached to the subject 10 and carried by the subject 10. During theperiod, the receiving device 3 further adds related information such asreception intensity information and reception time information at eachof the receiving antennas 4 a to 4 h at a timing when each image data isreceived to the image data received through the receiving antenna unit4. Then, the receiving device 3 stores these image data and the relatedinformation in the memory 33. After the imaging of the capsule endoscope2 is completed, the receiving device 3 is removed from the subject 10and connected to the image display device 5. Then, the receiving device3 transfers the image data and the related information stored in thememory 33 to the image display device 5. In FIG. 1, a cradle 3 a isconnected to a USB port of the image display device 5 and the receivingdevice 3 is set in the cradle 3 a, so that the receiving device 3 isconnected to the image display device 5.

The image display device 5 is configured by using, for example, aworkstation or a personal computer. The image display device 5 performspredetermined image processing on an image in the subject 10 acquiredthrough the receiving device 3 to generate an observation screen of apredetermined format and displays the observation screen.

FIG. 3 is a block diagram illustrating a schematic configuration of theimage display device 5. As illustrated in FIG. 3, the image displaydevice 5 includes an input unit 51, an image data acquisition unit 52, acontrol unit 53, a storage unit 54, and a display unit 55.

The input unit 51 is realized by an input device, such as a keyboard, amouse, a touch panel, and various switches. The input unit 51 inputs asignal according to an operation from outside by a user to the controlunit 53.

The image data acquisition unit 52 is an interface that can be connectedto a communication line such as USB, wired LAN, or wireless LAN, andincludes a USB port, a LAN port, and the like. In the first embodiment,the image data acquisition unit 52 functions as a data acquisition unitthat acquires the image data and the related information from thereceiving device 3 through an external device, such as the cradle 3 a,connected to the USB port and various communication lines.

The control unit 53 is realized by hardware such as a CPU and transmitsan instruction and data to each unit included in the image displaydevice 5 and controls the operation of the entire image display device 5based on a signal inputted from the input unit 51, the image dataacquired by the image data acquisition unit 52, and the like by readingvarious programs stored in the storage unit 54. In more detail, thecontrol unit 53 includes an image processing unit 531, a positioninformation acquisition unit 532, an index creation unit 533, an imageidentification unit 534, a determination unit 535, a candidate imageextraction unit 536, a feature point setting unit 537, and a displaycontrol unit 538.

The image processing unit 531 generates display image data by performingimage processing, such as white balance processing, demosaicing, colorconversion processing, density conversion processing such as gammaconversion, smoothing processing by denoising and the like, andsharpening processing by edge enhancement and the like, on image data ofa series of images obtained through the image data acquisition unit 52.Further, the image processing unit 531 performs predetermined imageprocessing such as average color calculation processing that calculatesan average color of each image based on the display image data.

The position information acquisition unit 532 detects a position of thecapsule endoscope 2 when an image is captured, that is, a position of anobject captured in an image, based on the reception intensityinformation when the receiving antennas 4 a to 4 h receive a wirelesssignal among the related information of the image data of each image.Hereinafter, the position of the capsule endoscope 2 while an image iscaptured is simply referred to as a capsule position. A detection methodof the capsule position is not limited to a method based on thereception intensity of the wireless signal, but it is possible to use,for example, a method that uses magnetic field disclosed in JP2009-226080 A and other various known methods.

The index creation unit 533 creates a position series index thatrepresents a relationship between an imaging sequence of a series ofimages acquired by the capsule endoscope 2 and the capsule position.FIG. 4 is a schematic diagram illustrating an example of the positionseries index. The position series index 100 is a two-dimensional arraytable in which image numbers that represent the imaging sequence ofimages are arranged according to a distance from a predeterminedreference position to the capsule position. As an example, the positionof the mouth of the subject 10 from which the capsule endoscope 2 isswallowed is set as the reference position.

The numerical values written in the uppermost row of the position seriesindex 100 illustrated in FIG. 4 are parameters representing a distancefrom the reference position. Hereinafter, the parameter that representsa distance from the reference position is simply referred to as adistance parameter. In the first embodiment, a distance (cm) from thereference position is defined as the distance parameter. The numericalvalues written in the second and lower rows of the position series index100 are the image numbers, and each image number is written in a columnof a distance parameter corresponding to the capsule position of theimage. For example, it is indicated that the image of the image number(i−6) is an image captured at a position corresponding to a distanceparameter 100. A distance between the capsule positions of the images ofthe image number (i−6) and (i−5) written in fields adjacent to eachother is 2 cm, and a distance between the capsule positions of theimages of the image number (i−5) and (i−4) written in fields adjacent toeach other with one field in between is 4 cm. How to create the positionseries index 100 will be described later. Hereinafter, an image to whichan image number i is attached is also referred to as an image (i).

When a selection signal to select one of a series of images is inputtedto the image identification unit 534 from the input unit 51 according toa user operation to the input unit 51, the image identification unit 534identifies image data corresponding to the selection signal asidentified image data. Hereinafter, an image corresponding to theidentified image data, that is, an image selected by the user, isreferred to as an identified image.

The determination unit 535 determines whether or not the identifiedimage that is identified by the image identification unit 534 is animage acquired in a section where the capsule endoscope 2 makes areciprocating movement in which the capsule endoscope 2 moves forwardand backward in the subject based on the position series index 100illustrated in FIG. 4.

In more detail, the determination unit 535 includes an identificationunit 535 a that identifies a plurality of images where the capsuleposition or an imaging time is within a predetermined range with respectto the identified image. The determination unit 535 determines that theidentified image is an image acquired in a section where the capsuleendoscope 2 makes the reciprocating movement when the plurality ofimages identified by the identification unit 535 a include two or moreimage groups which continue in a forward direction in time series aswell as continue in a forward direction in position series.

When the determination unit 535 determines that the identified image isan image acquired during the reciprocating movement of the capsuleendoscope 2, the candidate image extraction unit 536 extracts an imagethat is more appropriate as an image to set a feature point from amongimages acquired during the reciprocating movement. Hereinafter, theimage extracted by the candidate image extraction unit 536 is referredto as a candidate image.

The feature point setting unit 537 sets the identified image that isidentified by the image identification unit 534 or the candidate imagethat is extracted by the candidate image extraction unit 536 as an imagethat indicates a feature point and sets a capsule position of the imagethat indicates the feature point as a feature point according to asignal inputted from the input unit 51. Hereinafter, the image thatindicates the feature point is referred to as a feature point image.

The display control unit 538 causes the display unit 55 to display aseries of images in a predetermined format and causes the display unit55 to display the candidate image extracted by the candidate imageextraction unit 536 in a predetermined format based on display imagedata that is image-processed by the image processing unit 531.

The storage unit 54 is realized by a semiconductor memory such as aflash memory, a RAM, and a ROM, a recording medium such as an HDD, anMO, a CD-R, and a DVD-R, a writing/reading device that writes and readsinformation to and from the recording medium, and the like. The storageunit 54 includes an image data storage unit 541 that stores the displayimage data that is image-processed by the image processing unit 531 andstores the capsule position of each image in association with the imagedata, an index storage unit 542 that stores the position series indexcreated by the index creation unit 533, a feature point storage unit 543that stores information related to the feature point image, a settingcondition storage unit 544 that stores a condition used when extractinga candidate image, and a program storage unit 545 that stores a programand various information to operate the image display device 5 and causevarious functions to be performed.

Among them, the setting condition storage unit 544 stores a settingcondition of a section where the capsule endoscope 2 makes thereciprocating movement in the subject 10 and a condition when extractinga candidate image from a section where the capsule endoscope 2 makes thereciprocating movement. Hereinafter, the section where the capsuleendoscope 2 makes the reciprocating movement is simply referred to as areciprocating movement section. The setting condition of thereciprocating movement section includes conditions such as the length ofthe reciprocating movement section, in other words, the distance of thereciprocating movement section, the number of images acquired in thereciprocating movement section or the time required to move in thereciprocating movement section, and the position of the reciprocatingmovement section with respect to the identified image. As an extractioncondition of the candidate image, there is a condition that an imagecaptured when the capsule endoscope 2 passes through the reciprocatingmovement section for the first time is defined as the candidate image ora condition that an image captured when the capsule endoscope 2 passesthrough the reciprocating movement section for the last time is definedas the candidate image. The setting condition storage unit 544 stores apredetermined initial condition in advance as the setting condition ofthe reciprocating movement section and the extraction condition of thecandidate image. However, a user may overwrite a setting condition ofthe reciprocating movement section and an extraction condition of thecandidate image which the user desires by an operation using the inputunit 51.

The display unit 55 is realized by a display device such as a CRTdisplay, a liquid crystal display, and an EL display. The display unit55 displays a series of images, the candidate image extracted by thecandidate image extraction unit 536, the other related information, andthe like on a screen in a predetermined format under control of thedisplay control unit 538.

Next, an operation of the image display device 5 will be described. FIG.5 is a flowchart illustrating processing that transfers (downloads)image data acquired by examination using the capsule endoscope 2 fromthe receiving device 3 to the image display device 5.

First, in step S10, the receiving device 3 is set to the cradle 3 aconnected to the image display device 5 (see FIG. 1), and thereby theimage data is transferred from the receiving device 3 to the imagedisplay device 5. Alternatively, the image data may be transferred fromthe receiving device 3 to the image display device 5 through variouscommunication lines.

In the following step S11, the image processing unit 531 creates displayimage data by performing image processing such as white balanceprocessing, demosaicing, color conversion, density conversion,smoothing, and sharpening on each image corresponding to the image datatransferred from the receiving device 3. In this case, the imageprocessing unit 531 may perform image processing such as calculation ofaverage color of each image along with the image processing describedabove.

In the following step S12, the position information acquisition unit 532detects the capsule position of each image based on the receptionintensity information in the related information of the image data.

In the following step S13, the index creation unit 533 creates theposition series index based on the capsule positions of a series ofimages detected in step S12. Specifically, as illustrated in FIG. 4, theimage numbers of a series of images are sequentially inputted intocolumns of distance parameter corresponding to the capsule position ofthe image in ascending order. In this case, when the distance parameteris the same as that of the previous image number or the distanceparameter with respect to the previous image number changes, forexample, from increase to decrease or decrease to increase, a row of theposition series index 100 is added and thereafter the image numbers areinputted into the added row.

For example, in FIG. 4, when inputting the image number (i−1), while thedistance parameter (112) of the previous image number (i−2) is increasedfrom the distance parameter (108) of the further previous image number(i−3), the distance parameter (110) of the image number (i−1) isdecreased from the distance parameter (112) of the image number (i−2).In this case, the image number (i−1) and the following image numbers areinputted to an added row (second row). When inputting the image number(i+2), while the distance parameter (102) of the previous image number(i+1) is decreased from the distance parameter (104) of the furtherprevious image number (i), the distance parameter (104) of the imagenumber (i+2) is increased from the distance parameter (102) of the imagenumber (i+1). In this case, the image number (i+2) and the followingimage numbers are inputted to an added row (third row). Further, thedistance parameter (104) of the image number (i+3) is the same as thedistance parameter of the previous image number (i+2), so that the imagenumber (i+3) and the following image numbers are inputted to an addedrow (fourth row).

In the following step S14, the storage unit 54 stores the image data onwhich image processing is performed in step S11, stores the capsulepositions detected in step S12 in association with the image data, andfurther, stores the position series index created in step S13. Thereby,the transfer processing of the image data from the receiving device 3 tothe image display device 5 is completed.

Next, feature point setting processing on a series of images will bedescribed. FIG. 6 is a flowchart illustrating the feature point settingprocessing performed by the image display device 5. In the descriptionbelow, it is assumed that the position series index illustrated in FIG.4 is created and the position series index is stored in the storage unit54.

First, in step S20, the control unit 53 acquires the extractioncondition of the candidate image from the setting condition storage unit544. Specifically, the control unit 53 acquires either of the followingconditions: extracting an image when the capsule endoscope 2 “first”passes through as the candidate image and extracting an image when thecapsule endoscope 2 “last” passes through as the candidate image fromthe section where the capsule endoscope 2 makes the reciprocatingmovement.

In the following step S21, the control unit 53 acquires the settingcondition of the reciprocating movement section from the settingcondition storage unit 544. In the first embodiment, a case will bedescribed in which the reciprocating movement section is a range wherethe capsule position is ±Δ from the center which is the identifiedimage. In FIG. 4, as an example, is 4 cm.

In the following step S22, the control unit 53 reads the image data fromthe image data storage unit 541 and causes the display unit 55 tosequentially display a series of images acquired by the capsuleendoscope 2 in a predetermined format based on the image data.

FIG. 7 is a schematic diagram illustrating a display example of theseries of images acquired by the capsule endoscope 2. A display screenM1 illustrated in FIG. 7 includes an examination information displayfield m11 in which examination information such as examination ID andexamination date is displayed, a patient information display field m12in which patient information such as patient ID, patient name, and dateof birth is displayed, a main display area m13 in which a series ofimages are sequentially displayed in a pseudo moving image format, anoperation button group m14 used for operation of a reproductionoperation of the images displayed in the main display area m13, afeature point setting button m15 used to input an instruction to selectan image being displayed in the main display area m13 as a feature pointimage, a track display area m16 in which a track of the capsuleendoscope 2 in the subject is displayed, an average color bar m17, andan overview button m18 used to input an instruction to switch a displayformat of the series of images from the pseudo moving image format to astill image list display format. When a plurality of imaging units 21are provided to the capsule endoscope 2 (see FIG. 2) and the imaging canbe performed in a plurality of directions, a plurality of main displayareas m13 may be provided corresponding to these imaging units 21.

The feature point setting button m15 is used to select an image thatincludes a portion which a user wants to set as a feature point. When apredetermined pointer operation such as, a click is performed on thefeature point setting button m15 by using the input unit 51, a selectionsignal to select an image displayed in the main display area m13 at thistiming as a feature point image is inputted from the input unit 51 tothe control unit 53.

In the track display area m16, a track m19 where the capsule positionsassociated with images acquired by the position information acquisitionunit 532 are connected and a marker m20 that indicates the capsuleposition corresponding to the image being displayed in the main displayarea m13 are displayed. The user can roughly know the capsule positionof the image being displayed in the main display area m13 by referringto the position of the marker m20 on the track m19.

The average color bar m17 is a bar in which average colors calculated bythe image processing unit 531 for each image are arranged in a band-likearea in the arrangement order of the images, that is, in the order ofthe time of imaging. The average color bar m17 is provided with a sliderm21 that indicates a position corresponding to the image being displayedin the main display area m13. The user can roughly know the position inthe subject of the image being displayed in the main display area m13 byreferring to the position of the slider m21 in the average color barm17. It is possible to change the image displayed in the main displayarea m13 by moving the slider m21 along the average color bar m17 by apredetermined pointer operation using the input unit 51.

Reduced images m22 that are obtained by reducing the feature pointimages are displayed as thumbnails below the average color bar m17. Eachreduced image m22 is connected by a line to a corresponding position onthe average color bar m17.

In the following step S23, the control unit 53 determines whether or notthe selection signal to select the image being displayed in the maindisplay area m13 as a feature point image is inputted from the inputunit 51. When the selection signal of the image is not inputted (stepS23: No), the processing proceeds to step S38 described later.

On the other hand, when the selection signal of the image is inputted(step S23: Yes), the image identification unit 534 acquires the imagenumber of the image corresponding to the selection signal (that is, theidentified image) (step S24).

In the following step S25, the determination unit 535 reads the positionseries index from the index storage unit 542 and acquires the locationof the image number of the identified image in the position seriesindex. For example, when the image number of the identified image is i,the image number i in the position series index 100 illustrated in FIG.4 is located in the second row of the column of the distance parameter(104).

In the following step S26, the determination unit 535 sets areciprocating movement section for the identified image and extracts allrows where an image number is located in the reciprocating movementsection. For example, as illustrated in FIG. 4, when a range where thecapsule position is ±4 cm from the center, which is the identified imagei, is defined as the reciprocating movement section, a section from thecolumn of distance parameter (100) to the column of distance parameter(108) is defined as the reciprocating movement section. In this case,the determination unit 535 extracts the first to the fourth rows, wherethere is an image number in the above reciprocating movement section,from the position series index.

In the following step S27, the determination unit 535 acquires motioninformation of the capsule endoscope 2 in each row extracted in stepS26. Here, the motion information is information that indicates areciprocating motion in which the capsule endoscope 2 moves forward andreverse when the images corresponding to the image numbers written ineach row are acquired. The motion information includes “forward” thatindicates a movement in the forward direction, that is, a direction frommouth to anus, “reverse” that indicates a movement in the reversedirection, that is, a direction from anus to mouth, and “staying” thatis neither the “forward” nor the “reverse”. In FIG. 4, the staying isdescribed by a code “−”. The motion information can be discriminated bythe image numbers written in each row and the distance parameters.Specifically, when continuous image numbers are arranged in ascendingorder corresponding to the forward direction for each row of theposition series index, if the distance parameter corresponding to eachimage number increases, the motion information of the row is “forward”.On the other hand, when continuous image numbers are arranged inascending order corresponding to the forward direction, if the distanceparameter corresponding to each image number decreases, the motioninformation of the row is “reverse”. When only one image number isinputted in one row, the motion information is the staying “−”.

Specifically, in the case of FIG. 4, it is determined that the motioninformation of the first row that includes the image numbers (i−6) to(i−3) is “forward”, the motion information of the second row thatincludes the image numbers (i) and (i+1) is “reverse”, the motioninformation of the third row that includes the image number (i+2) is thestaying, and the motion information of the fourth row that includes theimage numbers (i+3) and (i+4) is “forward”. Thereby, it is possible toknow the motion of the capsule endoscope 2 when the images correspondingto the image numbers inputted into each row are acquired.

In the following step S28, the determination unit 535 determines whetheror not the motion information acquired in step S27 includes two or more“forward”. For example, in the reciprocating movement sectionillustrated in FIG. 4, the motion information of the first and thefourth rows is “forward”, so that there are two or more “forward”.

When there are not two or more “forward” (step S28: No), it isconsidered that the capsule endoscope 2 proceeds only in the forwarddirection in the reciprocating movement section. Therefore, in thiscase, the feature point setting unit 537 sets a feature point based onthe identified image, that is, the image selected by the user (stepS29). Specifically, the feature point setting unit 537 adds a flagindicating that the identified image is a feature point image to theidentified image and sets the capsule position of the feature pointimage as the feature point. Thereafter, the processing proceeds to stepS38 described later.

On the other hand, when there are two or more “forward” (step S28: Yes),it is considered that the capsule endoscope 2 makes a reciprocatingmovement or temporarily stays at the same position in the reciprocatingmovement section. In this case, there may be another image of the sameportion as that of the identified image or a portion close to theportion of the identified image. Therefore, the candidate imageextraction unit 536 extracts an image more appropriate as the featurepoint image as a candidate image.

In step S30, the candidate image extraction unit 536 determines whetheror not the extraction condition of the candidate image acquired in stepS20 is an image when the capsule endoscope 2 “first” passes through thesection where the capsule endoscope 2 makes a reciprocating movement.

When the extraction condition of the candidate image is the “first”(step S30: Yes), the candidate image extraction unit 536 acquires allthe image numbers included in a first row of the reciprocating movementsection, that is, all the image numbers included in a first forward path(step S31). On the other hand, when the extraction condition of thecandidate image is not the “first” (step S30: No), that is, when theextraction condition of the candidate image is the “last”, the candidateimage extraction unit 536 acquires all the image numbers included in alast row of the reciprocating movement section, that is, all the imagenumbers included in a last forward path (step S32).

In the following step S33, the candidate image extraction unit 536determines whether or not the image number of the identified image, thatis, the image number corresponding to the selection signal inputted instep S23, is included in the image numbers acquired in step S31 or S32.

When the image number of the identified image is included in the imagenumbers acquired in step S31 or S32 (step S33: Yes), the processingproceeds to step S29.

On the other hand, when the image number of the identified image is notincluded in the image numbers acquired in step S31 or S32 (step S33:No), the candidate image extraction unit 536 acquires image datacorresponding to all the image numbers acquired in step S31 or S32 fromthe image data storage unit 541 (step S34).

In the following step S35, the display control unit 538 causes thedisplay unit 55 to display an image corresponding to the image dataacquired by the candidate image extraction unit 536 in step S34 as acandidate image.

FIG. 8 is a schematic diagram illustrating a display example of thecandidate image. In a display screen M2 illustrated in FIG. 8, a windowm32 including a plurality of candidate images m31 is superimposed on thedisplay screen M1 illustrated in FIG. 7 and displayed. In the windowm32, a message display field m33 for a user and a cancel button m34 arealso displayed. In the message display field m33, a message urging theuser to select a feature point image from among the candidate images m31is displayed. When a predetermined pointer operation such as, a click isperformed on any one of the candidate images m31 displayed in the windowm32 by using the input unit 51, a signal to select the candidate imageon which the pointer operation is performed is inputted from the inputunit 51 to the control unit 53. When a predetermined pointer operationsuch as, a click is performed on the cancel button m34 by using theinput unit 51, a signal that instructs cancellation of the setting offeature point is inputted from the input unit 51 to the control unit 53.

In step S36, the feature point setting unit 537 determines whether ornot a signal to select any one of the candidate images m31 displayed inthe window m32 is inputted from the input unit 51. When the signal toselect any one of the candidate images m31 is inputted (step S36: Yes),the feature point setting unit 537 sets a feature point based on theselected candidate image (step S37). Specifically, the feature pointsetting unit 537 adds a flag indicating that the selected candidateimage is a feature point image to the selected candidate image and setsthe capsule position of the feature point image as the feature point.

On the other hand, when the signal to select any one of the candidateimages m31 is not inputted (step S36: No), that is, when a signal thatinstructs cancellation of the setting of the feature point is inputted,the processing proceeds to step S38.

In step S38, the control unit 53 determines whether or not there is animage to be displayed next. When there is an image to be displayed next(step S38: Yes), the processing returns to step S22. On the other hand,when there is no image to be displayed next (step S38: No), a series ofprocessing ends.

As described above, according to the first embodiment of the presentinvention, when an image acquired while the capsule endoscope 2 makes areciprocating movement is selected as a feature point image by a useroperation, if there is an image more appropriate as the feature pointimage in a series of images, the image is displayed on a screen as acandidate image, so that the user can set a more appropriate featurepoint by referring to the candidate image displayed on the screen.

Further, according to the first embodiment of the present invention, itis possible to correctly extract a section, where the capsule endoscope2 makes a reciprocating movement, with simple processing by using theimage numbers that indicate an arrangement order of the images and thedistance parameters that represent the capsule positions of the images.

Modified Example 1

Next, a modified example 1 of the first embodiment of the presentinvention will be described. FIG. 9 is a schematic diagram illustratinganother example of an index used when determining motion information ofthe capsule endoscope 2. An image number index 200 illustrated in FIG. 9is a two-dimensional array table in which the image numbers thatrepresent an imaging sequence of images are arrayed based on theincrease/decrease of distance from a predetermined reference position tothe capsule position of an image. The numerical values written in theuppermost row of the image number index 200 are parameters representingthe increase/decrease of distance from the reference position to thecapsule position. Hereinafter, the parameter that represents theincrease/decrease of distance from the reference position to the capsuleposition is simply referred to as an increase/decrease parameter. Thenumerical values written in the second and lower rows of the imagenumber index 200 are the image numbers. It is indicated that, among theimage numbers written in the same row, the greater the value of theincrease/decrease parameter, the longer the distance from the referenceposition to the capsule position.

In the image number index 200, a row is added when the distance from thereference position to the capsule position is the same as that in theprevious image, is changed from increase to decrease, or is changed fromdecrease to increase. For example, the image number (i−1) is inputtedinto a newly added row because the distance that has been increasingbegins to decrease. Further, the image number (i+2) is inputted into anewly added row because the distance that has been decreasing begins toincrease. Further, the image number (i+3) is inputted into a newly addedrow because the distance is the same as that of the previous imagenumber (i+2). In each row, a continuous image number is inputted in anadjacent field regardless of an absolute value of the distance from thereference position to the capsule position.

In the case of such an image number index 200, for example, thereciprocating movement section is set in a form of ±columns based on thekth column that includes the image number i of the identified image. InFIG. 9, k±2 columns are defined as the reciprocating movement section.Based on the reciprocating movement section set in this way, it ispossible to determine the motion information of each row and extract acandidate image in the same manner as in the first embodiment.

Modified Example 2

Next, a modified example 2 of the first embodiment of the presentinvention will be described. FIG. 10 is a schematic diagram illustratingfurther another example of the index used when determining the motioninformation of the capsule endoscope 2. A time series index 300illustrated in FIG. 10 is a two-dimensional array table in which animaging distance that represents a distance from a predeterminedreference position to the capsule position of each image is arrayedaccording to an imaging time period that represents an elapsed time froman imaging start time to an imaging time of the image. Here, when animaging frame rate is constant, the imaging time period corresponds tothe imaging sequence, so that the image number that represents theimaging sequence of each image can be used as a time parameter thatrepresents the imaging time period.

In the time series index 300, a row is added when the imaging distance(cm) is the same as that of the most previously imaged image, is changedfrom increase to decrease, or is changed from decrease to increase. Forexample, the imaging distance 110 cm corresponding to the time parameter(i−1) is inputted into a newly added row because the imaging distancethat has been increasing begins to decrease. The imaging distance 104 cmcorresponding to the time parameter (i+2) is inputted into a newly addedrow because the imaging distance that has been decreasing begins toincrease. Further, the imaging distance 104 cm corresponding to the timeparameter (i+3) is inputted into a newly added row because the imagingdistance is not changed from that of the previous image number (i+2).

In the case of such a time series index 300, for example, thereciprocating movement section is set in a form of ±Δ based on the timeparameter i of the identified image. In FIG. 10, i±4 is defined as thereciprocating movement section. By setting the reciprocating movementsection in this way, it is possible to determine the motion informationof each row and extract a candidate image in the same manner as in thefirst embodiment.

Modified Example 3

Next, a modified example 3 of the first embodiment of the presentinvention will be described. In the first embodiment described above,the position series index is created when the image data is transferredfrom the receiving device 3 to the image display device 5. However, theposition series index may be created at any time before the featurepoint setting processing is performed. For example, the position seriesindex may be created when a user sets the feature point or a user opensan image data file to observe an image. The same goes for a case inwhich the image number index 200 described in the modified example 1 orthe time series index 300 described in the modified example 2 is createdinstead of the position series index.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe first embodiment described above, by determining whether or not twoor more rows where the motion information is “forward” are included inthe reciprocating movement section with respect to the identified imageselected by a user operation, it is determined whether or not thecapsule endoscope 2 makes a reciprocating movement or is staying in thereciprocating movement section. However, the movement of the capsuleendoscope 2 may be directly determined from the reciprocating movementsection without individually determining the motion information of eachrow.

FIG. 11 is a flowchart illustrating feature point setting processingperformed by an image display device according to the second embodimentof the present invention. A configuration of the entire capsuleendoscope system and a configuration of the image display device of thesecond embodiment are the same as those of the first embodimentillustrated in FIGS. 1 to 3. Steps S20 to S26 illustrated in FIG. 11 arethe same as those in the first embodiment.

In step S41 following step S26, the determination unit 535 determineswhether or not the images included in the reciprocating movement sectioninclude two or more rows, that is, two or more image number groups, ineach of which the image numbers are arranged in the forward direction intime series, that is, in ascending order, and the image numbers arearranged in the forward direction in position series.

When the reciprocating movement section does not include two or morerows, in each of which the image numbers are arranged in the forwarddirection in time series as well as in the forward direction in positionseries (step S41: No), the processing proceeds to step S29. On the otherhand, when the reciprocating movement section includes two or more rows,in each of which the image numbers are arranged in the forward directionin time series as well as in the forward direction in position series(step S41: Yes), the processing proceeds to step S30. The processing ofsteps S29 to S38 are the same as those in the first embodiment.

As described above, according to the second embodiment of the presentinvention, it is determined whether or not the capsule endoscope 2 makesa reciprocating movement in a reciprocating movement section based onthe arrangement order of the image numbers included in the reciprocatingmovement section instead of determining the motion information for eachrow of the position series index 100, so that it is possible to simplifythe processing.

Also in a case in which the image number index 200 described in themodified example 1 or the time series index 300 described in themodified example 2 is created instead of the position series index, itis possible to determine the motion of the capsule endoscope 2 by thesame processing.

Third Embodiment

Next, a third embodiment of the present invention will be described. Acapsule endoscope system according to the third embodiment of thepresent invention includes an image display device 6 illustrated in FIG.12 instead of the image display device 5 illustrated in FIG. 1. FIG. 12is a block diagram illustrating a configuration of the image displaydevice according to the third embodiment of the present invention. Theimage display device 6 illustrated in FIG. 12 includes a control unit 61instead of the control unit 53 illustrated in FIG. 3. A configurationand an operation of each unit in the image display device 6 other thanthe control unit 61 and a configuration of the entire capsule endoscopesystem are the same as those of the first embodiment illustrated inFIGS. 1 and 2.

The control unit 61 includes a determination unit 611 instead of thedetermination unit 535 included in the control unit 53 illustrated inFIG. 3. The determination unit 611 includes a similarity determinationunit 611 a that determines a similarity between each of a plurality ofimages where the capsule position or the imaging time is within apredetermined range with respect to an identified image identified bythe image identification unit 534 and the identified image. When two ormore image groups which include an image that is determined to besimilar to the identified image by the similarity determination unit 611a and in which the image numbers are continuous are included in theplurality of images, the determination unit 611 determines that theidentified image is an image acquired in a section where the capsuleendoscope 2 makes a reciprocating movement. A configuration and anoperation of each unit in the control unit 61 other than thedetermination unit 611 are the same as those in the first embodiment.

FIG. 13 is a flowchart illustrating the feature point setting processingperformed by the image display device 6. Steps S20 to S25 illustrated inFIG. 13 are the same as those in the first embodiment.

In step S51 following step S25, the determination unit 611 sets areciprocating movement section for the identified image, extracts allrows where an image number is located in the reciprocating movementsection, and acquires image data corresponding to each image number fromthe image data storage unit 541.

In the following step S52, the determination unit 611 acquires imagedata of the identified image.

In the following step S53, the similarity determination unit 611 adetermines a similarity between the identified image and the otherimages based on the image data acquired in step S51 and S52. As a methodfor determining the similarity, various known methods can be applied.Specifically, Normalized Cross-Correlation (NCC), SSD (Sum of SquaredDifference), and SAD (Sum of Absolute Difference) by template matchingand the like are used as an index representing the similarity. It isindicated that the greater the value of NCC is, the higher thesimilarity between the images is, and the smaller the value of SSD andSAD is, the higher the similarity between the images is. Therefore, animage where the NCC is greater than or equal to a predetermined valueand an image where the SSD or the SAD is smaller than or equal to apredetermined value are determined to be similar to the identifiedimage.

In the following step S54, the determination unit 611 determines whetheror not there are two or more rows that include one or more image numbersof images similar to the identified image. In other words, it isdetermined whether or not there are two or more image groups whichinclude an image similar to the identified image and in which the imagenumbers are continuous.

When there are not two or more rows that include an image number of animage similar to the identified image (step S54: No), the feature pointsetting unit 537 sets a feature point based on the identified image (theimage corresponding to the selection signal inputted in step S23) (stepS29). Thereafter, the processing proceeds to step S38 described later.

On the other hand, when there are two or more rows that include an imagenumber of an image similar to the identified image (step S54: Yes), thecandidate image extraction unit 536 determines whether or not theextraction condition of the candidate image acquired in step S20 is animage when the capsule endoscope 2 “first” passes through the sectionwhere the capsule endoscope 2 makes a reciprocating movement (step S55).

When the extraction condition of the candidate image is the “first”(step S55: Yes), the candidate image extraction unit 536 extracts animage with an image number which is included in the first row of thereciprocating movement section, that is, the first forward path, fromamong the images similar to the identified image (step S56). On theother hand, when the extraction condition of the candidate image is notthe “first” (step S55: No), the candidate image extraction unit 536extracts an image with an image number which is included in the last rowof the reciprocating movement section, that is, the last forward path,from among the images similar to the identified image (step S57).

In the following step S58, the candidate image extraction unit 536causes the display unit 55 to display the image extracted in step S56 orS57 as a candidate image. The following steps S36 to S38 are the same asthose in the first embodiment described with reference to FIG. 6.

As described above, according to the third embodiment of the presentinvention, the candidate image is extracted based on the determinationresult of the similarity with the identified image, so that it ispossible to present a more appropriate candidate image to a user.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Acapsule endoscope system according to the fourth embodiment of thepresent invention includes an image display device 7 illustrated in FIG.14 instead of the image display device 5 illustrated in FIG. 1. FIG. 14is a block diagram illustrating a configuration of the image displaydevice according to the fourth embodiment of the present invention. Theimage display device 7 illustrated in FIG. 14 includes a control unit 71and a storage unit 72 instead of the control unit 53 and the storageunit 54 illustrated in FIG. 3.

The control unit 71 further includes a passing time period calculationunit 711 and an organ function determination unit 712 in addition to thecomponents of the control unit 53 illustrated in FIG. 3. The passingtime period calculation unit 711 calculates a passing time period of thecapsule endoscope 2 between feature points different from each other.The organ function determination unit 712 determines a function of anorgan in the subject based on the passing time period calculated by thepassing time period calculation unit 711. A configuration and anoperation of each unit in the control unit 71 other than the passingtime period calculation unit 711 and the organ function determinationunit 712 are the same as those in the first embodiment.

The storage unit 72 further includes a statistics storage unit 721 inaddition to the components of the storage unit 54 illustrated in FIG. 3.The statistics storage unit 721 stores statistics of the passing timeperiod of the capsule endoscope 2 in the subject for each type of organsuch as stomach, small intestine, and large intestine. It is morepreferable for the statistics storage unit 721 to accumulate statisticsof the passing time period of the capsule endoscope 2 in each organ foreach age, gender, and medical history of a patient who is the subject.As the statistics, any one of the average value, the mode value, and themedian value may be used. A configuration of each unit in the storageunit 72 other than the statistics storage unit 721 are the same as thosein the in the first embodiment.

Next, an operation of the image display device 7 will be described. Inthe same manner as in the first embodiment, the image display device 7has a function to additionally perform function determination of anorgan in the subject after performing transfer of image data from thereceiving device 3 (see FIG. 5) and setting of a feature point (see FIG.6). The image data transfer processing and the feature point settingprocessing performed by the image display device 7 are the same as thoseof the first embodiment. Alternatively, the feature point settingprocessing may be performed in the same manner as in the second and thethird embodiments.

FIG. 15 is a flowchart illustrating organ function determinationprocessing performed by the image display device 7. First, in step S60,the organ function determination unit 712 determines whether or not theentrance of the stomach and the entrance of the small intestine havebeen set as feature points on a series of images. When the entrance ofthe stomach and the entrance of the small intestine have not been set asfeature points (step S60: No), the processing proceeds to step S65described later.

On the other hand, when the entrance of the stomach and the entrance ofthe small intestine have been set as feature points (step S60: Yes), thepassing time period calculation unit 711 calculates the time period forpassing through the stomach of the capsule endoscope 2 (step S61). Thetime period for passing through the stomach is obtained by calculating adifference between the imaging time of the feature point image set asthe entrance of the stomach and the imaging time of the feature pointimage set as the entrance of the small intestine.

In the following step S62, the organ function determination unit 712reads a statistic of the time period for passing through the stomachfrom the statistics storage unit 721 and compares the time period forpassing through the stomach calculated in step S61 with the statistic.When the time period for passing through the stomach calculated in stepS61 is smaller than or equal to the statistic (step S63: No), theprocessing ends.

On the other hand, when the time period for passing through the stomachcalculated in step S61 is longer than the statistic (step S63: Yes), itis considered that a function (peristaltic motion) of the stomach of thesubject is weakened. In this case, the organ function determination unit712 causes the display unit 55 to display a message indicating that thefunction of the stomach is weakened (step S64). A specific example ofthe message is “The time period for passing through the stomach islonger than usual. The function of the stomach may be weakened.” and thelike.

In the following step S65, the passing time period calculation unit 711determines whether or not the entrance of the small intestine and theentrance of the large intestine have been set as feature points. Whenthe entrance of the small intestine and the entrance of the largeintestine have not been set as feature points (step S65: No), theprocessing ends.

On the other hand, when the entrance of the small intestine and theentrance of the large intestine have been set as feature points (stepS65: Yes), the passing time period calculation unit 711 calculates thetime period for passing through the small intestine of the capsuleendoscope 2 (step S66). The time period for passing through the smallintestine is obtained by calculating a difference between the imagingtime of the feature point image set as the entrance of the smallintestine and the imaging time of the feature point image set as theentrance of the large intestine.

In the following step S67, the organ function determination unit 712reads a statistic of the time period for passing through the smallintestine from the statistics storage unit 721 and compares the timeperiod for passing through the small intestine calculated in step S66with the statistic. When the time period for passing through the smallintestine calculated in step S66 is smaller than or equal to thestatistic (step S68: No), the processing ends.

On the other hand, when the time period for passing through the smallintestine calculated in step S66 is longer than the statistic (step S68:Yes), it is considered that a function of the small intestine of thesubject, specifically, the peristaltic motion, is weakened. In thiscase, the organ function determination unit 712 causes the display unit55 to display a message indicating that the function of the smallintestine is weakened (step S69). A specific example of the message is“The time period for passing through the small intestine is longer thanusual. The function of the small intestine may be weakened.” and thelike. Thereafter, the processing ends.

As described above, according to the fourth embodiment of the presentinvention, it is possible to accurately calculate the time period forpassing through each organ by the capsule endoscope 2 based on theappropriately set feature points. Therefore, when functional decline ofan organ is predicted from a comparison result between the time periodfor passing through an organ and the statistic, by notifying a useraccordingly, the user can perform image observation under considerationof the functional decline of the organ.

The present invention described above is not limited to the first to thefourth embodiments and the modified examples thereof, but variousinventions can be formed by appropriately combining a plurality ofcomponents disclosed in the embodiments and the modified examples. Forexample, the inventions may be formed by removing some components fromall the components described in each of the embodiments and the modifiedexamples or may be formed by appropriately combining componentsdescribed in different embodiments and modified examples.

In the above description, the first to the fourth embodiments of thepresent invention and the modified examples thereof are describedassuming a case in which a capsule endoscope is used in the medicalfield. However, the embodiments and the modified examples may be appliedin a case in which a capsule endoscope is used in fields other than themedical field.

According to some embodiments, it is determined whether or not an imageselected by a user is an image acquired while a capsule endoscope makesa reciprocating movement. When it is determined that the image selectedby the user is an image acquired while the capsule endoscope makes areciprocating movement, an image that is more appropriate as an image toset a feature point is extracted from a series of images, and the imageis displayed as a candidate image. Therefore, the user can easily set anappropriate feature point by referring to the candidate image.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

What is claimed is:
 1. An image processing device comprising: aprocessor comprising hardware, wherein the processor is configured to:identify an image selected according to a user input operation receivedby an input device, from among a plurality of first images acquired by acapsule endoscope, as an identified image; determine whether or not theidentified image is one of a plurality of second images acquired duringa reciprocating movement of the capsule endoscope; and in response tothe identified image being determined to be one of the plurality ofsecond images, extract an image acquired in a first forward path or alast forward path of the reciprocating movement from among the pluralityof second images.
 2. The image processing device according to claim 1,wherein the processor is further configured to: acquire a plurality ofcapsule positions, each capsule position being a position of the capsuleendoscope when each of the plurality of first images is acquired; anddetermine whether or not the identified image is one of the plurality ofsecond images based on an imaging sequence of the plurality of firstimages and the plurality of capsule positions.
 3. The image processingdevice according to claim 2, wherein the processor is further configuredto: identify a plurality of third images, each third image where thecapsule position or an imaging time is within a predetermined range withrespect to the identified image; and determine that the identified imageis one of the plurality of second images when the plurality of thirdimages include two or more image groups which continue in a forwarddirection in time series as well as continue in a forward direction inposition series.
 4. The image processing device according to claim 1,wherein the processor is further configured to: acquire a plurality ofcapsule positions, each capsule position being a position of the capsuleendoscope when each of the plurality of first images is acquired;identify a plurality of third images, each third image where the capsuleposition or an imaging time is within a predetermined range with respectto the identified image; determine a similarity between the identifiedimage and each of the plurality of third images; and determine that theidentified image is one of the plurality of second images when two ormore image groups, which include an image that is determined to besimilar to the identified image and in which image numbers arecontinuous, are included in the plurality of third images.
 5. The imageprocessing device according to claim 4, wherein the processor is furtherconfigured to extract an image determined to be similar to theidentified image from among images acquired in the first forward path orthe last forward path.
 6. The image processing device according to claim1, wherein the processor is further configured to control a display todisplay the image extracted by the processor.
 7. An image processingmethod comprising: identifying an image selected according to a userinput operation, from among a plurality of first images acquired by acapsule endoscope, as an identified image; determining whether or notthe identified image is one of a plurality of second images acquiredduring a reciprocating movement of the capsule endoscope; and inresponse to the identified image being determined to be one of theplurality of second images, extracting an image acquired in a firstforward path or a last forward path of the reciprocating movement fromamong the plurality of second images.
 8. A non-transitorycomputer-readable recording medium recording an image processing programfor causing an image processing device to execute: identifying an imageselected according to a user input operation, from among a plurality offirst images acquired by a capsule endoscope, as an identified image;determining whether or not the identified image is one of a plurality ofsecond images acquired during a reciprocating movement of the capsuleendoscope; and in response to the identified image being determined tobe one of the plurality of second images, extracting an image acquiredin a first forward path or a last forward path of the reciprocatingmovement from among the plurality of second images.